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I . INTRODUCTION 



As the computing needs of an organization expand, the 
decision to add to, or replace, existing computer systeias is 
important and may have long term effects on an organization. 
The impact can be financial, operational or both. The 
choice of computer systems, hardware and software will be a 
major factor in defining the information processing 
capabilities of an organization. The methods for selecting 
computer systems are varied and have evolved over the past 
20 years. 

Computer systems have become less expensive and, at the 
same time, significantly more powerful. Advances in data 
communication technology have led to the increased use of 
distributed data processing (DDP) and decentralized computer 
systems. In order to effectively use DDP and decentralized 
computing, computer systems throughout an organization must 
be compatible with each other. Data and files have to be 
transferadsle and usecd^le on all systems throughout an 
organization . 

Once a decision is made to acquire a new computer 
system, an organization needs to ensure that its selection 
is the best choice to meet its present and future computing 
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needs. If the selected computer system does not fulfill the 
organizational requirements there might not be another 
chance to make a better choice due to cost and time 
constraints . 

The emphasis of this thesis is on the evaluation and 
selection phases of a computer system acquisition process. 
The actual selection of a computer system is preceded by 
several procedural steps. The following is a generalization 
of those procedures: 

• Analyzing the requirements and computing needs of the 
organization . 

• Determining and defining the requirements for the 
computer system. 

• Sending the request for proposal (RFP) to qualified 
vendors . 

• Screening, evaluating, validating and comparing the 
proposals . 

• Selecting the computer system. 

From the multitude of computer systems, operating 
systems, software and peripherals, a user has to choose 
those products that best meet an organization's needs. The 
difficult task facing those responsible for the selection is 
to determine by what standards competing systems are 
compared and evaluated. 
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Traditional procedures have addressed the selection of 
individual computer systems. With DDP and decentralized 
computing becoming more prevalent, there is a need to apply 
new methods for selection. The concern is no longer the 
selection of a single system; it is the selection of a group 
of systems that work together and ene^Dle the sharing of 
files and data. 
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II. COMPUTER SELECTION PROCEDURES 



As long as there has been more than one computer system 
available to meet a user's requirements, there has been a 
need to select the one that best fits the job. The problem 
is how to make that selection. There have been a nximber of 
modaXa and methods proposed and used to select computer 
systems . The various methods used have concentrated on the 
individual stages in the selection process and on specific 
techniques for each stage. 

The selection process consists of several stages : 
analysis of needs, determination of requirements, request 
for proposals, evaluation of proposals and selection of a 
system. The first two steps are preliminary to the 
evaluation of the proposals, while the latter two are often 
blurred into a single step: evaluate the candidates and pick 
the best. 

The techniques for the evaluation of computer systems 
generally are either simple or sophisticated. The simple 
methodologies are basically intuitive, unsupported by any 
theory or lacking a systemized approach. The sophisticated 
methods employ an analysis of vendors' proposals with a 
ranking of the proposed computer systems. Following this 
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ranking, a hardware performance and capability analysis is 
performed on the most desirable alternatives. If the system 
that was ranked first fails to perform adequately then that 
system is rejected and analysis is carried out on the next 
highest ranked system. 

The most commonly used computer system evaluation 
methods are : 

Weighted Scoring. This commonly used approach attempts to 
overcome evaluation problems by combining objectivity with 
consideration of apparently non-quantif iable factors. 
Relative weights are pre-assigned to all system items 
considered important. Each competent system is subjectively 
rated with respect to each selected system item and then the 
overall score is computed for each vendor. The rating is 
siibjective: from "no good" to "entirely adequate". The 
system with the highest total score is considered most 
desirable. (Borovits and Zviran, 1987) 

The advantage of this method is that it is simple. It 
also enables one to perform sensitivity analysis of the 
results for changes in the importance weights of the 
attributes. Its disadvantage is that it is not normative, 
i.e. it is not based on a system of axioms expressing 
rational behavior rules expected of a decision maker. Thus, 
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it does not require an examination of assumptions of 
independence in the attributes and the absence of such an 
examination may result in deviations. (Shoval and Lugasi, 
1987) The following example demonstrates this: 

Two proposals, A and B, are examined according to two 
independent attributes, "vendor support" and "hardware 
performance" . If A and B receive the same score in the 
evaluation and if "vendor support" is evaluated as entirely 
adequate in proposal A and no good in proposal B while 
"hardware performance" is entirely adequate in B and no good 
in A, then the "hardware performance" of B may never be 
available because of its no good "vendor support". 

(Timmreck, 1973) 

This method does not allow for an examination of 
consistency of the evaluators. Due to its nature, the 
method does not consider risk and uncertainty. It should be 
noted that granting scores in every attribute is in itself a 
difficult task and influenced by subjective considerations. 
(Shoval and Lugasi, 1987) 

Multi -Attribute Utility Model. Shoval and Lugasi (1987) use 
of Keeny' s multi-attribute utility model allows an 
evaluation of the utility function of the attributes and the 
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calculations of their weights. It differs from other models 
in that it considers risk and uncertainty. 

The application of the model requires both utility and 
preference independence. Utility independence claims that 
for attributes XI... Xn the utility of attribute Xi does not 
depend on the remaining attributes . Preference independence 
claims that preference between every pair of attributes does 
not depend on the fixed level of remaining attributes. 

There are two variants, which differ in the way they 
refer to risk, the additive model and the multiplicative 
model. The application of either depends on a decision 
maker's attitude towards risk. 

In order to evaluate the utility function of the 
attributes and to calculate the weights/constants of the 
attributes, a gambling technique is used, a utility function 
of the Von-Neuman-Morgenstern type . 

Evaluation of the decision maker's utility function is 
based on the axioms of transitivity and continuity. If 
these axioms are satisfied, it is possible to form a utility 
function for the decision maker for each attribute. 

The multi-attribute utility model is normative, since it 
forces a decision maker to accept a set of axioms which 
express preferences and it also requires an examination of 
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independence assumptions before the model is applied. The 
model also ena±>les one to make sensitivity analyses. 

Compared with other models this model is more difficult for 
the decision maker to understand and apply, since 
determining a probability at which a decision maker is 
indifferent between alternatives is not always clear and 
acceptable. (Shoval and Lugasi, 1987) 

Cost-Effectiveness Ratio. This is simply a sub-category of 
the weighted scoring technique, where the cost of the entire 
computer system is divided by the sum of its points. The 
system having the highest total score is considered the most 
desirable. (Borovits and Zviran, 1987) 

Cost -Value. This approach uses cost as the sole measure. 
Cost and benefits are assigned to the desirable features of 
each proposed system. The sum of these values is then 
s\jbtracted from the total cost of the system. The result 
represents the cost of the system to meet to meet the 
mandatory requirements. The system with the lowest cost is 
the most desirable. (Timmreck, 1973) 

Requirement -Co sting Technique. Here the estimated cost- 
savings benefits are assigned to a set of desirable 
features. Those not part of the proposed system are then 
added to the total cost of the system. In addition, if a 
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desirable feature is offered at an additional cost that is 



less than the estimated benefit, the incremental cost is 
added to the system cost . The system with the lowest 
resulting cost is considered the most desirable. 

(Borovits, 1984) 

Dynamic Approach. This approach employs a projected work 
load growth trend and cost-effectiveness ratios of each 
vendor's proposed product line. The underlying assumption 
is that it would be feasible to install a system for as 
little as one year, but only if it will be replaced by a 
compatible system from the same manufacturer's product line 
and replacement accords with the dictates of work load. 

Using the projected work load trend and measurement of 
system capacity, a schedule of replacement is worked out . 

The cost of each system (as determined by one of the methods 
described above) over the period of its employment is then 
discounted with respect to an estimate of cost of capital 
and its expected future life. The product line with the 
lowest discounted present cost is the most desirable. (Ein- 
Dor, 1977) 

Present Value Analysis. The objective here is to determine 
which system will cost the least and benefit the 
organization most. The net present value of the proposed 
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system should be determined by discounting all cash flows 
associated with it, using an organization's cost of capital 
as the discount rate. The benefits projected by the 
proposed new system are the net cash inflows. The system 
with highest net present value is the most desirable. 
(Borovits and Zviran, 1987) 

Efficient -Frontier Model. In this model, the preferred 
system is that which is dominant in all the attributes 
considered according to a decision maker's preference. 

This method enables one to screen out alternatives that 
are inferior in all attributes or are identical in some and 
inferior in ac least one. The alternatives remaining after 
this screening form an "efficient frontier" . There remains 
the problem of selecting a preferred alternative; this 
requires other methods. Therefore, this model is not 
sufficient. It is applicable, at most, in performing 
initial screening. (Shoval and Lugasi, 1987) 

Lexicographical Ordering. In this model, alternatives are 
rank-ordered according to a dominant attribute, the most 
important one, e.g. CPU power. Clearly, this is possible 
only if a dominant attribute exists and cannot be traded off 
with others. This model can, at most, help in the initial 
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screening of alternatives which are deficient in an 
important attribute but it cannot choose the best . (Ahituv 
and Neumann, 1986) 



11 



III. COMPUTER-FAMILY SELECTION 



The selection procedures being discussed focus on the 

selection of a group of systems that will work together, 

sharing files and data. This group of systems has been 

called a computer-family and defined as: 

Con^utttrs of the same type, coneieting of several models 
from the same manufacturer's product line, ranging from 
microconputer to mainframe, with full con^atibility in 
the operating system and the system's software, to 
enable transfer of application software from one family 
member to another without change. 

(Borovits and Zviran, 1987) 

An example of a computer-family is Digital Equipment 
Corporation's (DEC) VAX 884Q mainframe computer, VAX 6220 
minicomputer and Microvax 3600 super-microcomputer. Another 
example consists of a Prime 6650 mainframe computer, a Prime 
4450 minicomputer and a Model 4050 super-microcomputer. 

Selecting a computer-family is a more complex process 
than selecting a computer system. The process will have a 
lasting effect on an organization because there will be one 
computer-family from which current and future hardware 
acquisitions will be made. This will help ensure uniformity 
and compatibility in information processing throughout an 
organization . 
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For DDP and decentralized computing the compatibility 
obtained in hardware and software will preclude the systems 
integration problems encountered while using mismatched 
systems. The benefits of system-wide compatibility are 
exemplified by the ability to transfer application software 
from one family member to another using a common operating 
system. 

A number of methods are used for computer system 
selection. Some of the concepts presented in the existing 
methodologies form the basis for a methodology to deal with 
computer-family selection. The procedures presented here 
are a ten-step generic evaluation and selection methodology. 
(Borovits and Zviran, 1987) 

Figure 1 portrays a work flow diagram of this process. 
The work flow diagram highlights the processes that occur 
after an analysis of an organization takes place and 
systems' requirements document are developed. 

The stages in this process are: 

Step 1. Identification of possible vendors and 
manufacturers: After studying the requirements document it 
may be determined which manufactures or vendors may be able 
to provide products that will meet an organizational needs. 
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step 1. Identification of possible 
vendors and manufacturers. 

\ 

Step 2. Primary elimination of 
irrelevant candidates . 

» 

Step 3. Determination of 
mandatory requirements . 



Step 4. Examination of vendors' compliance 
with mandatory requirements . 



Step 5. Setting quantitative and qualitative 
criteria and respective weighting scales. 

♦ 

Step 6. Writing the RFP to be addressed 

to selected vendors. 

■ 

♦ 



Step 7. Receiving, comparing and analyzing bids. 
Step 8. Concluding final list of vendors. 



Step 9 . Performance of hardware 
and software benchmarks . 

♦ 

Step 10. Drawing final conclusions 
and selection of best computer-f amily . 



Figure 1. Selecting a computer-family: a work flow 
diagram. (Borovits and Zviran, 1987) 
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A list of possible vendors can be generated and used in 
future stages. 

Step 2. Preliminary elimination of irrelevant 
candidates: Further study of an initial list of 

manufacturers and vendors should reveal apparently 
unsuitedjle candidates. They should be pared from the 
original list to produce a manageaJjle list of potential 
(relevant) vendors. Reasons for elimination are for 
unsuitcQjility in meeting requirements or because they do not 
have a complete computer-family. 

Step 3 . Determination of mandatory requirements : From 

the systems requirement document and from organizational 
policy certain features and characteristics of a computer- 
family are identified as prerequisites for further 
consideration. These are criteria that must be met by any 
potential computer-f amily . 

Step 4. Examination of vendor's compliance with 
mandatory requirements: On the basis of mandatory 
requirements, a questionnaire is formulated and addressed to 
all relevant vendors. Responses to this questionnaire 
provide a selection committee with basic information about 
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proposed computer -fauni lies and each product line. The 
responses are screened to determine if they meet mandatory 
requirements. The vendors remaining after this elimination 
procedure constitute a mailing list for requests for 
proposals (RFP) . 

Step 5. Setting quantitative and qualitative criteria 
and respective weighting-scales: The variability of the 
capabilities and performance characteristics of each element 
(hardware, software, service) of a computer system is very 
large, and the number of permutations is considerable. A 
properly constituted set of quantitative and qualitative 
criteria is critical for the acquisition of a satisfactory 
computer-family. Furthermore, vendors are expected to 
propose not a single computer system but rather a wide 
product line of systems working together. This raises a 
problem of evaluating and comparing proposals. Borovits and 
Zviran (1987) suggested that the quantitative selection 
criteria be applied to at least the following five issues, 
each of which has its own partial list of guidelines. 

[Fig. 2] 
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HARDWARE (for each model within the fcunily) 

* memory size (minimum, maximum, units of expansion) 

* disks (number of drives, minimum and maximum capacity) 

* tapes (maximum nxamber of drives, density) 

* diskette readers (diskette size, density) 

* terminals 

* printers 

* backup features 

* miscellaneous 

SOFTWARE (for the entire fcunily) 

* system software (uniformity of operating system, compilers, 
utilities, etc., throughout the family) 

* programming languages (compilers and interperters, options 
and extensions, adherence to standards) 

* development tools (existence of application generators, re- 
port generators, full screen editor, debugging aids, macro 
facilities, etc.) 

* additional software (electronic-spreadsheet, word-processor, 
electronic-mail, DBMS, etc.) 

* ability to transfer software within the family (source mod- 
ules, load modules, procedures) without change or conver- 
sion 

COMMUNICATION 

* communication protocols 

* existence of LAN, possibilty of connection to LAN 

* possibilty to transfer files, prograuns and procedures through 
communication 

* network transfer rate 

CONVERSION 

* possibilty to convert application software (programs and 
procedures) from present system to proposed computer- 
family 

ENVIRONMENT 

* environment requirements (such as airconditioning, temper- 
ature, humidity, etc.) 



Figure 2. List of criteria. (Borovits and Zviran, 1987) 
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Certainly not less important, though non-quantif iable, 
are qualitative criteria such as: 

• how widespread is the use of the proposed computer- 
family 

• vendor support, such as training, maintenance philosophy 
and practice 

• software houses specializing in the proposed computer- 
faunily 

• current users' opinions and vendor's reputation 
Once a detailed and complete set of data relating to all 
criteria is collected, weights are assigned to all items, 
both qualitative and quantitative, indicating their relative 
importance. These weights are fundamental to the evaluation 
process, explicitly indicating that the objective of the 
evaluation process is to ensure the selection of a computer- 
fcimily that best fits the needs (present and future) of an 
organization . 

Step 6 . Writing the RFP to be addressed to selected 
vendors: The primary advertising medium is the RFP, which 

is issued by a procuring organization and contains the 
system requirements. Typically, the RFP will include a 
sununary list of specific requirements according to which the 
vendors will be asked to write their proposals, describing 
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how they will meet each requirement. The RFP should consist 
of two major parts, the first relating to each model within 
the computer-fcunily and the second to the computer-f amily as 
a uniform entity. The second part is more general in nature 
and concentrates upon such issues as system software, 
conversion of present application software to the new 
computer-family, environment, etc. The RFP should be 
addressed to those vendors satisfying mandatory 
requirements . 

Step 7. Receiving, comparing and analyzing bids: The 

problem of analyzing and comparing bids is particularly 
important, since it is more than possible that none of the 
computer-families under consideration may exhibit a clear 
dominance over others. This stage is, therefore, 
fundcimental to the interim evaluation and selection process 
and should be accorded an appropriate measure of time and 
effort. It is required that the bids be submitted in 
writing and it is expected that they will conform to the 
style indicated in the RFP, so that the selection process 
will not be affected by style of expression and use of 
selling techniques. 
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The first step in the process of analyzing bids to 
subjectively rate each competing system with respect to each 
evaluation criterion. A score is assigned to each item. 

The weighted score is then computed for each proposed 
system. The next stage is the most problematic since there 
are no methods or guidelines for comparing computer- 
fcunilies. The proposed method consists of the following 
steps : 

• Five categories of computers are established: 

microcomputers (single station) 
super -microcomputers 
minicomputers 
super -minicomputers 
mainframe computers 

These categories represent differences in computing power 
and other major hardware characteristics. 

• Relative weights are assigned to each category according 
to its importance for an organization. 

• Each model within each computer- family is assigned to a 
category on the basis of criteria such as memory size, 
maximum nximber of terminals, maximum disk capacity and 
relative performance. 

• For each criterion, a computer model, or models, is 
selected within each category, which has dominance over 
its competitors and the score of 100 is assigned to it. 
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All other models of the remaining computer-families are 
then respectively an subjectively scored on a scale of 
0 - 100. 

• The total score is calculated. The score attained for 
each criterion is the average weight calculated for all 
five categories. 

• A comparative table is drawn up [Fig. 3] giving the 
scores attained by each family for each criterion. 

• The final score achieved by each family is calculated. 



Step 8. Drawing up a final list of vendors: On the 

basis of the final scores attained by each computer-family a 
selection committee is able to disqualify irrelevant 
computer-families, and select up to three or four vendors 
most likely to succeed. These computer families are then 
further tested to ensure they have the proper capabilities 
and characteristics. 

I 1 1 1 1 

I Criteria I Vendor A I Vendor B I Vendor C I 

Crit . 1 



1 Crit . 2 1 


1 


1 




1 Crit . 3 1 

1 1 

1 . f 

1 . 1 

1 . 1 

1 1 


1 

1 

1 

1 

1 

1 


1 

! 

1 

1 

1 





Figure 3 . Outline of comparative table . 

(Borovits and Zviran, 1987) 



21 



step 9 . Benchmarks for performance of hardware and 
software: Study of the written proposals is sufficient for 

evaluating fulfillment of certain requirements. In such 
cases, benchmarking must be performed. A benchmark, in the 
context of this discussion, is a set of live tests designed 
to examine the computer systems (hardware and software) 
proposed in response to an RFP . Selection of criteria to be 
tested may be performed according to their importance for an 
organization using the relative weights already assigned. 

The purpose of these benchmarks is to verify a system' s 
characteristics before drawing final conclusions. 

Step 10. Final conclusions and selection of the best 
computer-family: After benchmarks have been performed and 

all essential characteristics of a proposed computer-family 
have been deemed satisfactory, a selection committee will 
review and reconsider the relevant scores assigned to each 
competing computer-family. 

Finally, a committee will pick the best as the one 
recommended to be an organization's computer-family. The 
recommendations will then be submitted to an organization's 
management for approval and adoption. A problem faced by 



22 



those involved in the selection process is how to compare 
criteria and how to prioritize them according to their 
importance to the decision making process. There will be a 
large nvimber of criteria, some quantifiable and others non- 
quantif iable, whose importance to the selection process will 
be compared with each other. 
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IV. THE ANALYTICAL HIERARCHY PROCESS 



The methodology developed by Borovits and Zviran (1987) 
for computer-family selection discussed briefly the process 
of assigning relative weights to each category of computer 
in a family being evaluated (Step 5) . The process continues 
with each criterion assigned a relative weight so that it 
can be evaluated and scored. In a case study by Borovits 

and Zviran (1987) , the assigning of the relative weight and 

the actual scoring of computers-f amilies was done 
subjectively . 

Using subjective weighting and scoring can reduce the 
overall effectiveness of the process. There is little 
precision and it may be difficult to achieve replication 
using subjective weighting and scoring. 

An objective weighting and scoring system can improve 
this process. An objective weighting system should be 
characterized by consistency in the assignment of weights. 
Consistency is a factor that can be computed and compared to 

a standard. The standard is somewhere less than perfect 

consistency and greater than intolerable inconsistency. If 
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a standard is exceeded, this indicates randomness in the 
assignment of values. Should this occur, the criteria 
should be re-evaluated. (Saaty, 1980) 

An objective method that can be adapted for the 
comparison of a large number of objects, as is the case in 
comparing computer-faimilies, is the Analytical Hierarchy 
Process (AHP) developed by Saaty (1977) . 

The AHP can properly assess the importance of a large 
number of interacting factors, develop priorities among the 
factors and choose a best alternative (Saaty, 1980) . 

Seidmann and Arbel (1985) suggested an application of the 
AHP to the process of microcomputer selection. A large 
number of or attributes were used to compare microcomputers. 
The use of AHP allows the determination of both weights and 
scores for each attribute for each alternative, using 
matrices to perform pairwise comparisons between 
alternatives. Once weights and scores are obtained, the 
final score of each alternative is calculated according to a 
weighted scoring technique. 

In order to determine the weights of n attributes, a 
decision maker fills up a matrix with dimensions n x n in 
which a pairwise comparison is made between every two 
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attributes. (Fig. 4) Thus, in each cell (i,j) in the 
matrix, a decision maker expresses the relative importance 
of attribute i with respect to attribute j. (It is only 
necessary to fill up half of the matrix since a is the 
reciprocal of a Then, an eigenvector of the matrix is 

calculated for a maximxim eigenvalue. The eigenvector is 
normalized so that the total sum of its elements is 1. The 
values of this eigenvector constitute the attribute weights. 
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To determine the attribute scores for each alternative, 
a decision maker fills a matrix of pairwise comparisons 
between alternatives. Altogether, n matrices are filled 
(one for each attribute) . The dimension of every matrix is 
the niomber of alternatives. Here, too, a normalized 
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eigenvector is calculated and the elements of each such 
vector express the scores of the alternative for each 
attribute . 

Pairwise comparisons are accomplished with a relative 
numerical scale. Saaty (1977), in agreement with G.A. 
Miller's 1956 experiments proving individuals cannot 
simultaneously compare more than seven objects (plus or 
minus two) at the same time without being confused, chose a 
niamerical scale of 1 - 9. Using this scale, 1 expresses 
identity between two objects being compared and 9 expresses 
an absolute preference of one over the other. [Fig. 5] 

The Eigenvector method enables one to examine a decision 
maker's consistency, using appropriate measures (in essence, 
if a decision maker is consistent in a certain matrix, the 
maximum eigenvector of the matrix equals the order of that 
matrix) . On the other hand, this method ignores 
independence between attributes. It also does not consider 
risk and uncertainty. Nor does it enable one to perform 
sensitivity analyses. (Shoval and Lugasi, 1987) 

The AHP removes the guess work from the aasessinent of 
the relative importance of various criteria. The AHP can 
assist in analyzing several variables simultaneously. A 
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hierarchy is established so that sub-criteria relevant to a 
main criteria can be prioritized and comparisons can be made 
of minute details. (Saaty, 1980) 



Intensity of 
Importance 


Definition 


Explanation 


1 


Ecjual inpoirtanca of one 
element over another 


Two elements contribute 
equally to the property 


3 


Weak irrportance of one 
element over another 


Experience and judgment 
slighty favor one element 
over another 


5 


Essential or strong inpor- 
tance of one element over 
another 


Experience and judgment 
strongly favor one element 
over another 


7 


Demonstrated in^ortance of 
one element over another 


An element Is strongly 
favored and its dominance is 
demonstrated in practice 


9 


Absolute iiT^ortance of one 
element over another 


The evidence favoring one 
element over another is of the 
the highest possible order of 
affirmation 


2, 4, 6, 8 


Intermediate values between 
two adjacent judgments 


Comprimise is needed 
between two judgments 


Raclrocals 


If activity 1 has one of the 
preceeding numbers assigned 
to it when con^ared with 
activity j, then j has the 
reciprocal value when com- 
pared with i 





Figure 5 . 



Saaty's comparison table. 



(Saaty, 



1982) 
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V. PROPOSED METHODOLOGY 



Applying the AHP to Step 5 of Borovits and Zviran's 
methodology (Fig. 1) makes the resolution of ranking and 
weighting alternatives less arbitrary. 

In selecting a computer- family, specific attributes, 
capabilities, and perfoirmance characteristics, collectively 
known as criteria, are used as the basis for comparison 
between the competing alternatives. In Step 5 of Borovits 
and Zviran' s methodology, criteria are identified, 
prioritized and weighted according to their significance and 
value in the selection process. This procedure yields a 
maximum value for each criterion. Using the maximum values 
for all of the criteria, the highest possible score for each 
computer-family is computed. The maximum attainable scores 
in Step 5 are used in Step 7 . 

The AHP allows a decision maker to objectively create a 
prioritized and weighted list of criteria. At each level of 
the hierarchy, every criterion is compared to all the others 
in its group, on a one-to-one basis. Using the scale and 
descriptions from Figure 5, a score for each pairwise 
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comparison is obtained. These scores are inserted into a 
matrix to compute the weight of each criterion by Saaty's 
eigenvalue method. The weight of each criterion is used 
with the weights of the criteria that are above it in the 
hierarchy to compute a maximiam value . 

To facilitate the incorporation of the AHP into Step 5 
of Borovits and Zviran's methodology (Fig. 1), setting 
quantitative and qualitative criteria and respective 
weighting scales, the prioritization of all criteria is 
realized by completing six sub - steps . 

The sub steps are listed below in the order they are to 
be accomplished. 

5.1. - Prioritize the overall importance of qualitative 

versus c[uantitative criteria. 

5.2. - Select applicable computer categories. Each 

category is evaluated separately. 

5.3. - Select criteria for each computer category. 

5.4. - Select sub-criteria for each criterion until 

there are no more sub-criteria. 

5.5. - Prioritize and weight all categories, criteria, 

and sub-criteria. 
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5.6. - Calculate the absolute weights for all criteria 
and sub-criteria. 

In Step 7, the process of receiving, comparing and 
analyzing bids is comprised of two sub-steps; 

7.1. - Assign each relevant model of computer from a 

proposed computer-family to a category 
according to established criteria. 

7.2. - Evaluate each computer- family in accordance with 

the criteria established in Step 5. 

The advantage in applying the AHP to this methodology is 
that greater objectivity is achieved. No longer will an 
individual, or group, have to compare a large number of 
criteria and rank them, including the assignment of a 
relative value for each object. All comparisons are done 
one to one. The ranking and weighting of the criteria is 
computable and replicable. 



stop 1 . 



Identification of possible vendors and 
manufacturers . 



Stop 2. Primary elimination of irrelevant candidates. 

Stop 3. Determination of mandatory requirements. 

Stop 4. Examination of vendors' compliance with 
mandatory requirements. 



Stop 5. Setting quantitative and qualitative criteria 
and respective scales . 



5.1. Prioritize overall importance of qualitative 
and quantitative criteria. 

5.2. Select applicable computer categories. Each 
category is evaluated separately. 

5.3. Select criteria for each computer category 

5.4. Select sub-criteria for each criterion until 
there are no more sub-criteria. 

5.5. Prioritize and weight all categories, criteria 
and sub-criteria. 

5.6. Calculate the absolute weights for all criteria 
and sub-criteria. 



Stop 6. Writing the RFP to be addressed to selected 
vendors . 

Stop 7. Receiving, comparing, and analyzing bids. 



7.1. Assign each relevant model of computer from a 
proposed computer-family to a category according 
to predetermined criteria. 

7.2. Evaluate each computer-family in accordance 
with criteria established in Step 5. 



Stop 8. Drawing up a final list of vendors. 



Stop 9. Performance of hardware and software benchmarks. 
Stop 10. Drawing final conclusions and selection of best 



Figure 6. Updated workflow diagram. 
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VI. DEVELOPING SELECTION CRITERIA AND SCALE 



A. SELECTING AND PRIORITIZING CRITERIA 

In order to select a computer-family that best fulfills 
its requirements, an organization must designate the 
qualities that will be used to compare the computer- 
families. These qualities, or characteristics, are called 
selection criteria. 

Criteria that are used to evaluate computer-families are 
either quantifiable or non-quantif iable . The qualities that 
are not quantifiable are referred to as qualitative criteria 
and are evaluated subjectively. Those characteristics that 
are quantifiable and measurable by an established standard 
are called quantitative criteria, and are evaluated 
objectively . 

Criteria used in the evaluation process will be placed 
at different levels in a hierarchy. At the top of the 
hierarchy is the "Total Evaluation of a Computer -Family" . 

The second level consists of the division of qualitative and 
quantitative criteria. The next level is the first group of 
selectible criteria, in this case categories of computers. 
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At the lowest level are criteria that are elements of a 
criterion at the next higher level in the hierarchy. The 
criteria at each level, which are the descriptors of a 
criterion of the next higher level, are ranked in order of 
importance and scaled by their relative weight . This is the 
process of prioritizing. 

The process of prioritization will be conducted using 
the Analytical Hierarchy Process developed by Saaty(1980) . 
This uses a one to one comparison of all the items at the 
same level, producing a relative ranking and weighting of 
each criterion at that level in the hierarchy. These are 
the actions that take place during Step 5 [Fig. 6] . 

Step 5.1. Prioritize the overall importance of 
cfualitative versus quantitative criteria. Because there are 
both qualitative and quantitative criteria used in the 
evaluation process, prioritization begins with the 
determination of the weights, or percentage of the total 
score, for each of these groups of criteria. This is a 
subjective decision made without the use of the AHP . It is 
the second level in the hierarchy. 

Step 5.2. Select applicable computer categories. Each 
category is evaluated separately. Prioritization continues 
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with the selection of the applicable computer categories for 
the computer- family and the determination of the relative 
weight of each category. Computer category is the third 
level in the hierarchy. The categories from which to choose 
are : 

• Mainframe computer 

• Super-minicomputer 

• Minicomputer 

• Super-microcomputer 

• Microcomputer 

Step 5.3. Select criteria for each computer category. 

For each computer category designated in Step 5.2, choose 
the appropriate main categories of criteria that can be used 
in the evaluation process. The main categories of criteria, 
taken from the list in Figure 2, are: 

• Hardware 

• Software 

• Communication 

• Conversion 

• Environment 

These categories of criteria are the fourth level in the 
hierarchy . 
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step 5.4. Select sub-criteria for each criterion until 
there are no more sxib-criteria . For each main category of 
criteria. Step 5.3, in each main category of computer, Step 
5.2, select the appropriate sub-criteria. If a criterion 
can be further described by sub-criteria, continue the 
process of subdivision. Subdivision may occur down three, 
four or even five more levels. 

In selecting the quantif iadsle criteria, care must be 
taken to select relevant items and to not continue 
subdividing a criterion just because it is possible to do 
so. The criteria selected should be for valid items of 
comparison so that when the evaluation of the computer- 
families is made, and there is a difference in score between 
computer-families, the difference is valid and meaningful. 

A detailed list of quantitative criteria, which starts 
with the main categories of criteria at Level 4 in the 
hierarchy, is shown in Figure 8. For criterion that can be 
siobdivided, the sub-criteria are listed in hierarchy Level 
5, Level 6, and Level 7. 

The qualitative criteria, even though they are evaluated 
much more subjectively, can also be prioritized using 
Saaty's procedures. A detailed list of qualitative criteria 
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that may be used in comparison of computer-families is also 
shown in Figure 8. These criteria are at level 3 and level 
4 in the qualitative side of the hierarchy. The hierarchy 
is shown in Figure 7. 
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Hierarchy of criteria 



Level 4 Level 5 


Level 6 Level 7 


Hardware 

Memory 


Main 

Capacity 

ExpandcUaility 

Speed 

RAM 

Capacity 

Expandcibility 


Disks 


Number of 
drives 
Capacity 


Diskette 

Readers 


Size 

Number 

Density 


Terminals 


Monitors 

Monochrome 

Color 

Graphics abilities 
(CGA,EGA,VGA) 

Keyboards 

Voice 


Printers 


Color 

Graphics 

Laser 

Size 


Backup 


Size 

Speed 



Figure 8.1. Detailed List of Criteria, Hardware 
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Level 4 



Level 5 



Level 6 



Software 



Figure 8 



System 

software 

Operating system uniformity 

Utilities 

Compilers 

Compatibility 

Ability to modify 

Documentation 



Prograunming 

languages 

Compilers 
Interpreters 
Adherence to standards 
Options and extensions 



Development 

tools 

Application generators 
Report generators 
Full screen editors 
Debugging aids 
Macro facilities 

Additional 

software 

Word processors 
Spreadsheets 

Database Management Systems 
E-Mail 

Software 

transferability 



.2. Detailed List of Criteria, Software 
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Level 4 



Level 5 



Level 6 



Communication 

Communication 

protocols 

Local area 
networks 

Capacity (Nvimber of 
terminals) 

Maximiam distance 
Transmission medium 
Speed 

Wide area 
networks 

Transmission medium 
Internet capability 

Conversion 

Integration 
of current 
programs 

Integration 
of data 
files 

Environment 

New building 
requirements 

Special 

conditions 

Temperature 

Humidity 



Figure 8.3. Detailed List of Criteria, Communication, 
Conversion, and Environment. 
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Level 3 


Level 4 


Vendor 

Support 


Training 

Maintenance 

Philosophy 

Practice 


Vendor' s 
reputation 


User opinion 

Trade journal evaluations 



Manufacturer' s 
reputation 

User opinion 

Trade journal evaluation 

Number of users 
of computer-family 

Software companies 
specializing in 
this computer-family 



Figure 8.4. 


Detailed List of Qualitative Criteria 
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step 5.5. Prioritize and weight all categories, 
criteria and sub-criteria. For each category of computer, 
the appropriate criteria and sub-criteria are listed. After 
all the criteria are listed, they, along with the computer 
categories are prioritized and weighted using Saaty's 
eigenvector method. [Fig. 9] 

As described earlier, Saaty' s method requires a pairwise 
comparison of those criteria that are located at the same 
level in the hierarchy. Every criterion being compared is 
rated against every other criterion in the saime group. A 
value is obtained, based on the values from Figure 5, and 
inserted into the matrix. The size of the matrix should be 
limited to 7 +/- 2 items (Saaty, 1982) . The matrix 
generates a relative weight that is given to each category 
or criterion. This is an eigenvector; it has a decimal 
value or percentage of 1. The total value of all the 
weights generated for each group being compared is 1 or 
100 %. 

B. PRIORITIZING AND WEIGHTING: A DETAILED EXAMPLE 

After prioritizing the collective qualitative and 
quantitative criteria, Step 5.1, and then selecting the 
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applicable computer categories and appropriate criteria and 
sub-criteria, Steps 5.2 - 5.4, a decision maker is ready to 
begin computing priorities . 

In this example, three categories of computers, selected 
in Step 5.2, are placed in a matrix [Fig 9] . The three 
categories selected are Mainfreune computer, Minicomputer, 
and Microcomputer. To determine the relative weight for 
each category, a decision maker has to compare each category 
to the other two categories, one at a time, and select the 
number 1 to 9 which best represents the intensity of 
importance of one category to the other. The scale, shown in 
full in Figure 5, is presented here, in its abbreviated 
form . 



Intensity of 
Importance 

1 

3 

5 

7 

9 

2,4, 6, 8 



Definition 
Equal importance . 

Weak importance of one over another. 

Essential or strong importance. 

Demonstrated importance . 

Absolute importance . 

Intermediate values between the two 
adjacent judgments. 
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The numbers that are inserted into the matrix compare 
the item on the left with the item in the top row and 
represent the value from the scale cd^ove. If the item on 
the top is considered more important than the item on the 
left, a fraction, the reciprocal value of the comparison, is 
entered into the matrix. 

Scunple values have been entered into the matrix, 

[Fig. 9.1] Using the sample values an eigenvector will be 
computed for each computer category. The eigenvector will 
be used as the relative weight of the computer category. 



1 

1 

1 


Hierarchy Level 3 


1 

1 

1 


1 


1 Main- 


1 


1 Computer 


Category 1 fraune Mini 

1 


Micro 1 


1 Mainframe 


1 1 


3 1 


1 Mini 


1 1/5 1 


1/4 1 


1 Micro 


' 1/3 4 


1 1 
1 



Figure 9.1 The Initial Matrix 



The matrix is then normalized by adding the values in 
each colximn, then dividing each entry in each column by the 
total of that column. A new matrix with normalized values 
is created. [Fig. 9.2] 
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1 

1 

1 Computer Category 
1 


Main 
1 Frame 


Mini 


1 

Micro 




1 Mainframe 


1 1 


5 


3 




1 Mini 


1 1/5 


1 


1/4 




1 Micro 
1 


1 1/3 


4 


1 




1 Column total 
1 


1.533 


10 


4.25 

1 






\ 








1 


Main 




1 




1 Computer Category 
I 


1 Fraune 


Mini 


Micro 




1 Mainfreime 


1 0.652 


0.500 


0.706 




1 Mini 


1 0.131 


0.100 


0.059 




1 Micro 
1 


1 0.217 


0.400 


0.235 




Figure 9 . 2 The 


Normalized 


Matrix 






The last step is to average over the rows by adding the 


values in each row of the normalized matrix and dividing the 


rows by the number of entries 


1 in each. 


This number is 


the 


eigenvalue and is the weight 


that is given to that category. 



[Fig. 9.3] 



46 



Computed 

Relative 

Weight 



.652 + .500 + .706 



Mainframe 





3 






.131 + 


.100 + 


.059 






3 






.217 + 


.400 + 


.235 






3 




1. 



= .6193 



.2840 

1.0000 



Figure 9.3 Computing the Eigenvalue 



The total relative weight for all computer categories is 
equal to 1 or 100%. 

Consistency of the values must be taken into account in 
computing the eigenvectors using the calculated consistency 
index (Cl ) , dividing the Cl by the random value of 
consistency from a table. The random value is dependent on 
the mamber of items in the matrix. The resultant is the 
consistency ratio (CR) . The CR should be 10% or less, 
indicating judgments in comparing items in the matrix were 
not randomly made. (Saaty, 1982) 
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The matrices listed in Appendix A have listed in them 
criteria and s\ib-criteria from hierarchy Level 4 down 
through Level 7. For this example, only the first criterion 
from the previous level will have its s\ib-criteria listed in 
the next lower level. The numbers along the top row of each 
matrix refer to the corresponding numbered item on the left 
side of the matrix. The weight, in the last column, is the 
relative weight, the eigenvalue, of the criterion nauned in 
that row, in the left colxamn. 

For this example, the Mainframe Computer category is 
used. Its relative weight has already been calculated. 

[Fig. 9.3] The main categories of criteria are selected, and 
relative weights are computed for each of these criteria. 

The first matrix determines the relative weights a decision 
maker places on each of the main categories of criteria for 
each of the computer categories. The decisions made here 
have an effect on the importance of the decisions made lower 
in the hierarchy. For example, if hardware turns out to be 
worth 75% in the evaluation of minicomputers, the remaining 
categories of criteria are worth only 25%, no matter how 
many sub-criteria are used in the evaluation process. This 
is another reason not to subdivide criteria unnecessarily. 
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step 5.6. Calculate the absolute weights for all 



criteria and sub-criteria. Following the method proposed by 
Borovits and Zviran (1987 ) , a scoring system that makes 100 
points the maximum score attainable for a computer-f aunily , 
will be used in conjunction with the AHP . 

The cdDsolute weight for a criterion is computed by 
multiplying its relative weight by the relative weight of 
each of its predecessors in the hierarchy [Fig. 10], or by 
the absolute weight of its immediate predecessor. When a 
computer-family is evaluated the maximum score available for 
any criterion is its cibsolute weight. A list of absolute 
weights will be shown on an evaluation sheet. Appendix B 
used in conjunction with the scoring of the competing 
computer-families, in Step 7, 

An example of how an absolute weight is calculated is 
shown in Figure 10. A sample value is used for the relative 
weight of quantitative criteria. The relative weight for 
computer categories, mainframe computer was computed in 
Figure 9. 

The process is demonstrated in Figure 10 and is 
completed for each criterion in the hierarchy of a computer- 
fcunily. The absolute weight for the criteria whose relative 
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weights were calculated in Appendix A are shown on the 
Vendor Evaluation Form in Appendix B. 



1 Computer- 
1 Family 
1 Total Weight 


Quantitative Mainframe 1 
Criteria Computer 1 
Relative Weight Relative Weight 1 


1 = 1.00 


= 0.80 = 0.6913 1 



Absolute Weight for Mainframe Computer = 

(1.00) X (0.80) X (0.6913) = 0.4954 



Figure 10. Computing an absolute weight of a 
criterion 
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VII. THE EVALUATION PROCESS 



After the process of prioritizing the criteria has been 
completed, the process of evaluating each of the competing 
computer-fcunilies begins. In Borovits and Zviran's (1987) 
methodology this is accomplished as part of Step 7. 

In response to the request for proposal (RFP) that was 
sent out to vendors, bids for proposed computer-families 
will have been received. These bids then have to be 
analyzed and evaluated with scores assigned. In Steps 8, 9, 
and 10, the highest rated computer- families, on the basis of 
score, are selected for further evaluation and testing and, 
ultimately, one will be selected as the winner. 

The scope of Step 7 is the initial evaluation and 
scoring of the proposed computer-fcimilies, submitted in 
response to the RFP. As stated previously, this is 
completed by two sub-steps . 

Step 7.1. Assign each relevant model from a proposed 
computer-family to a category, according to predetermined 
criteria . In Step 5.2, computer categories were selected. 
The decision maker chose the computer categories that met an 
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organization's needs. Each relevant computer model from the 
proposed family will be placed in the appropriate computer 
category and will be evaluated by the criteria already 
selected for that category. 

The determination as to which category a computer will 
be placed is based on, but not limited to, such factors as 
CPU, memory or disk drives. The decision as to what factors 
will constitute placement into a particular category will 
have been determined when the categories were selected in 
Step 5.2. 

Step 7.2. Evaluate each computer-family in accordance 
with the criteria established in Step 5. Using the 
evaluation form. Appendix B, each computer-family is 
evaluated, criterion by criterion, with the result posted on 
the form. The total for the computer- fcimily is then 
computed and a final score is established. 

The maximxam score possible for any criterion is its 
cibsolute weight. The score assigned and posted on the form 
is on a scale of 0-1. Complying with the criterion 
completely is scored as 1, anything less is a decimal value 
less than 1. The posted score is multiplied by the absolute 
weight to yield the eODSolute score for that criterion. All 
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of the absolute scores are added to yield the final score 
for the computer-family. Only the lowest level criteria are 
scored. The maximum total score is 100. When this process 
has been completed for each of the computer-fcimilies, an 
objective comparison based on the final score may be made. 
The three or four highest rated computer-fcimilies may then 
be evaluated further by operational and benchmark testing. 

In Appendix B, the relative weights computed in Appendix 
A have been inserted into the evaluation sheet. Using those 
relative weights, the absolute weights for all criteria 
cited in the excunple have been computed. 
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VIII. CONCLUSION 



The process of selecting a computer-family is a complex 
procedure. The goal for a decision maker, responsible for 
selecting a computer- family, is to select the correct line 
of products for an organization. Because of the complexity 
of the selection process, a formalized methodology makes the 
process more objective. 

Using the methodology presented here, along with 
methodology developed by Borovits and Zviran (1987) , an 
objective selection process is created. The process allows 
the designation of appropriate criteria to be used in the 
selection of a computer-family. The appropriate criteria 
are those selected by an organization trying to select a 
computer-family, based on an evaluation of its needs. 

Once criteria have been selected, they are objectively 
prioritized and weighted, establishing their net value and 
absolute weight in an overall evaluation of a computer- 
family. Each computer-family is evaluated and scored 
separately, but in accordance with the prioritized and 
weighted criteria. 
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In spite of the careful process of prioritizing and 
weighting, total objectivity can not be achieved. 

Individuals comparing criteria will still be making 
subjective judgments as to the relative importance of one 
criterion versus another. This cannot be avoided. 

Subjectivity in the comparison of criteria may be 
counteracted by checking for consistency. The ability to 
measure the consistency in the comparison of criteria adds a 
check and balance to the process not previously present. 

By following the procedures developed in this thesis, 
the process of selecting a computer-family is made reliable 
and objective. The end product of this process, a computer- 
family that best meets the needs of an organization, may be 
chosen with the knowledge that the correct computer-family 
was selected. 
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APPENDIX A 



Sample values, using the guidelines from Figure 5, have 
been inserted into the matrices. Using the process in Step 
5.5, each matrix was normalized. The relative weight listed 
in the last column is the eigenvalue of the criterion in the 
first column in that same row. 



] 

Computer Category 


jevel 3 

12 3 


Relative 

Weight 


1 . Mainframe 

2. Mini 

3 . Micro 


15 3 

1/5 1 1/4 

1/3 4 1 


0.6193 

0.0964 

0.2842 

1.0000 



Level 4 



Main Categories Relative 



of Criteria (Mainframe) 


1 


2 


3 


4 


5 


Weight 


1 .Hardware 


1 


1/2 


5 


3 


6 


0.3024 


2 . Software 


2 


1 


5 


3 


7 


0.4119 


3 . Communication 


1/5 


1/5 


1 


1/4 


3 


0.0775 


4 .Conversion 


1/3 


1/3 


4 


1 


4 


0.1656 


5 . Environment 


1/6 


1/7 


1/3 


1/4 


1 


0.0426 

1.0000 
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Relative 



Hardware 


1 


2 


3 


4 


5 


6 


Weight 


1 .Memory 


1 


3 


6 


2 


5 


4 


0.3633 


2 . Disks 


1/3 


1 


3 


1/3 


3 


5 


0.1663 


3. Diskette Readers 


1/6 


1/3 


1 


1/5 


1/3 


1/3 


0.0420 


4 .Terminals 


1/2 


3 


5 


1 


3 


5 


0.2631 


5. Printers 


1/5 


1/3 


3 


1/3 


1 


3 


0.0986 


6 . Backup 


1/4 


1/5 


3 


1/5 


1/3 


1 


0.0667 



Relative 



Memory 


1 


2 


Weight 


l.Main 


1 


1/3 


0.25 


2. RAM 


3 


1 


0.75 



Level 7 








Relative 


Main Memory 


12 3 


Weight 


1. Capacity 


1 1/2 1/3 


0.1593 


2 . Expandibility 


2 1 1/3 


0.2518 


3 .Speed 


3 3 1 


0.5889 






1.0000 






Relative 


RAM 


1 2 


Weight 


1. Capacity 


1 1/4 


0.2 


2 .Expandibility 


4 1 


0.8 



APPENDIX B 



Vendor Evaluation Sheet R«iativ« Abaoiuto Absoiuto 











Nalght 


Waight 


Scora 


Scora 


Total 


1.000 


1.000 






Quantitative 


.8000 


.8000 








Mainfraune 


.6193 


.4954 








Hardware 


.3024 


.1498 








Memory 


.3633 


.0544 










Main 


.2500 


.0136 












Capacity 


.1593 


.0022 












Expandibility 


.2518 


.0034 












Speed 


.5889 


.0080 










RAM 


.7500 


.0408 












Capacity 


.2000 


.0082 












Expandibility 


.8000 


.0326 








Disks 


.1663 


.0249 










Number of drives 














Capacity 












Diskette Readers 


.0420 


.0063 










Size 














Number 














Density 
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Vendor Evaluation Sheet Relative Absolute Absolute 

Mainframe Hardware (cont'd) weight weight score score 





Terminals 


.2631 


.0394 










Monitors 














Monochrome 












Color 














Graphics abilities 












Keyboards 














Voice 












Printers 


.0986 


.0148 










Color 












Graphics 










Laser 










Size 












Backup 


. 0667 


.0010 








Size 










Speed 










Software 


.4119 


.2041 








System software 














Op. system uniformity 












Utilities 










Compilers 










Compatibility 










Ability to modify 










Documentation 
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Vendor Evaluation Sheet Relative Absolute Absolute 

Mainframe Software (cont'd) weight weight score score 



Programming languages 












Compilers 










Interpreters 










Adherence to standards 










Options and extensions 










Development tools 












Application generators 










Report generators 










Full screen editors 










Debugging aids 










Macro facilities 










Additional software 












Word processors 










Spreadsheets 










Database mgmt . systems 










E-Mail 










Software transferability 
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Vendor Evaluation Sheet Roiativ* Absolut* Absolut* 

Mainframe (cont'd) Walght W*ight Scor* Scor* 



Communication 


. 0775 


.0384 








Communication protocols 










Local area networks 














Capacity (# of terminals) 












Maximum distance 










Transmission medium 














Speed 












Wide area networks 














Transmission medium 










Internet capadDility 












Conversion 


.1656 


.0821 










Integrate current progms . 










Integration of data files 












Environment 


.0426 


.0211 










New building requirements 












Special Conditions 














Temperature 












Humidity 
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